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Abstract 

This paper summarizes the method of the Computer Sciences Corporation Flight Dynamics Operation (FDO) 
quality assurance approach to support the National Aeronautics and Space Administration Goddard Space Flight 
Center Flight Dynamics Support Branch (Code 553). Historically, a strong need has existed for developing 
systematic quality assurance using methods that account for the unique nature and environment of satellite Flight 
Dynamics mission support. Over the past few years, FDO has developed and implemented proactive quality 
assurance processes applied to each of six phases of the Flight Dynamics mission support life cycle: system and 
operations concept, system requirements and specifications, software development support, operations planning 
and training, launch support, and on-orbit mission operations. Rather than performing quality assurance as a final 
step after work is completed, quality assurance has been built in as work progresses in the form of process 
assurance. Process assurance activities occur throughout the Flight Dynamics mission support life cycle. The FDO 
Product Assurance Office developed process checklists for prephase process reviews, mission team orientations, 
in-progress process reviews, and end-of-phase audits. This paper will outline the evolving history of FDO quality 
assurance approaches, discuss the tailoring of CSC’s process assurance cycle procedures, describe some of the 
quality assurance approaches that have been or are being developed, and present some of the successful results. 


1.0 Background 

Since 1988, the Computer Sciences Corporation (CSC) Flight Dynamics Operation (FDO) has supported the 
National Aeronautics and Space Administration (NASA) Goddard Space Flight Center (GSFC) Flight Dynamics 
Support Branch (Code 553) as part of the CSC System, Engineering and Analysis Support (SEAS) contract. In 
January 1991, the FDO Product Assurance Office (PAO) performed its first formal audit with the Solar, 
Anomalous, and Magnetospheric Particle Explorer (SAMPEX) mission team members at the end of their system 
installation and acceptance phase. The audit was based on the SEAS System Development Methodology (SSDM) 
Standards and Procedures (S&Ps). As a result of this session, the PAO and SAMPEX team members realized that 
better audit results could have been achieved if the audit had been preceded by a coaching session. Since that first 
audit session, the goal and method of FDO audits have been redirected to assist mission team members in 
performing their mission support as the work progresses by identifying quality checkpoints and quality criteria and 
by reviewing the process rather than checking the end results. Consequently, the PAO entered into process 
reengineering with the FDO technical and management personnel in order to build quality into the work processes. 
(The terms product assurance and process assurance are used interchangeably throughout this paper.) Later in the 
stage of the FDO’s audit coaching or proactive quality assurance (QA) process development, the CSC SEAS 
process assurance cycle (PAC) procedures became available and were tailored to enhance the FDO process. The 
SEAS PAC comprises a series of reviews and audits before, during, and after each phase of the life cycle. In fact, 
the FDO has contributed to the SEAS PAC procedure development by providing FDO experiences and review 
comments for the draft SEAS PAC procedures. In FDO, the technical staff prefer the in-progress process review 
(IPPR) over the in-progress process audit (IPPA) by the PAO because it fosters a spirit of cooperative problem 
solving via process reviews, rather than giving the technical staff the impression that the PAO is checking on 
them. 


This work was supported by the National Aeronautics and Space Administration (NASA)/Goddard Space Flight Center 
(GSFC), Greenbelt, Maryland, under Contract NAS 5-31500. 
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2.0 Introduction 

The process assurance activities continue throughout the Flight Dynamics mission support life cycle, which 
comprises six phases: 

• System and Operations Concept 

• System Requirements and Specifications 

• Software Development Support 

• Operations Planning and Training 

• Launch Support 

• On-Orbit Mission Operations 

For convenience, the term life cycle phase is used here; however, in reality, a specific process such as requirements 
and specifications may go well beyond the requirements and specifications phase, which is normally considered 
complete at system requirements review (SRR). Likewise, all other adjacent phases often overlap, particularly 
during a rapid prototyping process for which a traditional waterfall model is not suitable. Further studies are 
required to model PAC procedures for such a prototyping process and also for the anticipated changes to a 
workstation support environment. 

Note that those phases that follow the requirements and specifications phase differ from the traditional system 
development life cycle phases. This difference shows the unique nature of the FDO mission support life cycle. For 
example, during the software development support phase, the FDO analysts support software development via 
walkthroughs and respond to developer questions after the system requirements and specifications documents are 
delivered. The CSC Software Engineering Operation (SEO) performs development in support of GSFC Code 552, 
Software Engineering Branch. 

Prior to the publication of this paper, no known published works have thoroughly described the QA processes for 
the entire life cycle of the GSFC Flight Dynamics mission. A great deal of effort has been expended within FDO to 
develop a set of systematic and proactive QA approaches tailored to the unique nature and environment of Flight 
Dynamics mission support. First, CSC FDO consolidated the existing QA activities and developed a process where 
those activities occur before the start of and/or during each mission phase in a proactive manner. The proactive QA 
activities are skills training, weekly internal mission reviews, document QA plans, dry runs, prephase preparation, 
mission team meeting (combined with just-in-time training for the upcoming phase), IPPRs, and prelaunch 
readiness reviews. Next, FDO PAO consulted with experienced staff to develop QA checklists for the six FDO 
mission support life cycle phases. Checklists are used to guide discussion at the IPPRs and LCPAs. The FDO just- 
in-time (JIT) training materials for the various FDO mission support life cycle phases and FDO document QA 
checklists were used for the phase IPPR and end of LCPA checklists. 

FDO’s strategy for producing quality products and services consists of the previously mentioned consolidated QA 
activities, techniques of coaching/training, lessons learned and feedback for process improvement, defect causal 
analysis, identification of quality checkpoints and associated quality criteria, identification of S&Ps and 
development of appropriate guidelines or S&Ps, and innovation. Each mission team within the FDO is responsible 
for developing and documenting improvements and continuing good practices based on the lessons learned during 
day-to-day activities or during the generation of products and services. The vehicle used to deploy this approach is 
the FDO PAC procedure described in the following sections. The FDO PAC procedure is supplemented by 
maintaining a project lessons learned database, FDO lessons learned database, DCA database, reports, and 
electronic files or bulletin boards. Communication is promoted among FDO projects on tools and methodologies 
that may contribute to improved quality and productivity. Methodology effectiveness is monitored and evaluated 
for improvement. 
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3.0 FDO PAC Procedure 


The FDO quality goals and PAC procedure are described in this section. 

3.1 FDO Quality Goals 

The goal of the FDO’s quality assurance effort is to maximize the quality of FDO products and services. This 
occurs by maximizing the quality and consistency of the technical processes used and detecting any problems in 
processes or quality early in the FDO mission support life cycle phases. Corrective actions are taken as early as 
possible to ensure that all deliverable FDO products and services are of high quality. Documents and reports 
produced are quality assured using the FDO Product QA Plan with the baseline quality criteria. 

3.2 FDO PAC Procedure 

The purpose of the FDO PAC procedure is to ensure that task teams have a consistent understanding of FDO 
mission support processes, products, and S&Ps before the start of each project and mission support life cycle phase. 
The PAC also shifts the emphasis of product assurance personnel from the traditional role of product inspector to 
that of process advisor (i.e., the technical personnel are responsible for ensuring the quality of a product or 
service.) In addition, use of the PAC helps the technical staff and the PAO identify and improve process-related 
weaknesses as early as possible in the life cycle. Early identification of items that should be changed is a most 
effective cost reduction/avoidance method. Finally, the PAC provides a standard, efficient, and repeatable 
mechanism for encouraging process improvement. The SEAS PAC applied to FDO comprises the following five 
activities: 

• Documentation of approach 

• Prephase Process Review (PPR) 

• Phase Orientation Meeting (POM) 

• In-Progress Process Review (IPPR) 

• Life cycle phase audit (LCPA) 

Figure 1 depicts the flow of the PAC process in each phase. 


Flight Dynamics Mission Support Life Cycle Phases 


Previous Phase 

Present Phase 

Next Phase 

4 4 

4 

► 


PPR POM IPPR LCPA 

(JIT Training (and lessons learned 

with lessons learned) analysis) 

Figure 1. FDO Process Assurance Cycle (PAC) Flow Before/During/After a Phase 


The FDO documentation of approach consists of project or task implementation plan, work package plan, and 
mission support life cycle phase activity plan. The PPR is an informal meeting of the project leader or mission 
manager with the PAO to review and update the approach (the project- or mission-specific checklist) for the 
upcoming life cycle phase. The PPR is intended to ensure that the activities, products, and events that are expected 
to occur during the upcoming life cycle phase are all identified and have corresponding S&Ps. The PPR should 
take place a few weeks before the start of the life cycle phase. POM is to ensure that all mission or project team 
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members have a consistent understanding of the S&Ps to be used, the processes that are to be followed, and the 
products and services to be delivered during the current life cycle phase. The POM is an opportunity for providing 
JIT training and lessons learned from previous missions for task or mission team members. Depending on the 
duration of the phases for a given mission or project, it may be appropriate to conduct a POM covering multiple 
phases. The IPPRs ensure that reviews are performed for task implementation plan, work package plan, and 
mission support phase activities. The IPPR provides an assessment of the mission or project team members' 
understanding and conformance to the S&Ps and provides processes that have been designated for use on the task 
through the mission- or project-specific checklist. The IPPRs occur approximately 15 to 25 percent into the 
calendar schedule of the task or mission support life cycle phase. This is early enough to correct any weaknesses 
detected in the process. The IPPR can occur more than once during the phase depending on the results of the first 
IPPR. At the conclusion of each phase, an LCPA is conducted to ensure that all the requisite activities have 
occurred and that it is appropriate to proceed to the next life cycle phase. Lessons learned data are collected and 
analyzed for training and process improvement during this audit. 

The FDO adaptation of the PAC procedure uses process checklists for each of the mission support life cycle phases. 
The FDO Mission Support Process Group and the FDO PAO developed these checklists because the SSDM S&Ps 
do not address the FDO’s mission support phases (Reference 1). The checklist for a given phase provides a list of 
the SSDM and FDO S&P applicable to the products and services that are developed and/or delivered during that 
phase. The process checklists may be tailored to meet the particular needs of a project. 

4.0 Flight Dynamics Mission Support Life Cycle Phases and Key Processes 

Figure 2 depicts the FDO mission support life cycle phases, reviews, and process overlaps. For convenience, 
numbers are assigned for the life cycle phases in Figure 2a. The activities supported by CSC SEO are identified in 
the right side of the figure. The reviews (Figure 2b) include system and operations concept review (SOCR), system 
requirements review (SRR), preliminary design review (PDR), critical design review (CDR), operations readiness 
review (ORR), and division operations readiness review (DORR). The processes of adjacent life cycle phases 
actually overlap as shown in Figure 2c. This figure also shows the relative effort of a mission team over time, but 
process assurance effort is about the same level throughout the life cycle. 

Key activities dominate all others for a given life cycle phase. The associated numbers help map activities between 
the GSFC Flight Dynamics mission life cycle phases and the CSC FDO life cycle phases. The key processes 
characterizing each phase are summarized as follows: 

• System and Operations Concept Process (1) 

- Investigate existing systems to fulfill anticipated mission support requirements using trade-off studies. 
If no existing system satisfies the requirements, a new system must be developed. 

• Requirements and Specifications Process (2) 

- Review and expand the high-level requirements in the detailed mission requirements (DMR) 
documents; identify items related to orbit and attitude. This process will result in additional 
refinements and further detailing of the Flight Dynamics support requirements. 

- Create a detailed Flight Dynamics support requirements table and clearly establish traceability of 
requirements. Continue to maintain the table as a quick reference to critical Flight Dynamics mission 
support requirements. 

• Software Development Support Process (3,4,5) 

- Respond to questions from the software development team. Specifications modifications may be 
needed as a consequence of the questions. 
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Figure 2. GSFC Flight Dynamics support life cycle consisting of CSC FDO - (and SEO-) supported 
life cycle phases (a), reviews after each phase (b), and process overlaps (c) 
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• Operations Planning and Training Process (6) 

- Develop mission support procedures. These procedures are standardized sets of ordered activities that 
may include software execution and other functions such as requesting, transmitting and archiving 
data, and logging. 

- Develop mission support timeline and operational scenarios (for example, scenarios for orbit 
acquisition, attitude acquisition, sun sensor calibration, and spacecraft ascent maneuvers). Each 
scenario consists of a collection of standardized procedures designed to satisfy mission requirements at 
the defined milestones in the timeline. 

- Participate in project simulations where the emphasis is on the interdivisional interfaces. 

• Launch Support Process (7) 

- Support early mission activities by monitoring spacecraft orbit and attitude, and data related to its 
health and safety. Plan and execute spacecraft orbit ascent maneuvers and attitude maneuvers. 
Validate onboard computer (OBC) attitude and orbit compared to the ground attitude and orbit. 

- Support calibration activities by planning and executing attitude sensor calibration maneuvers, if 
needed. Process calibration telemetry data and estimate sensor biases and misalignments. Send the 
results to the flight operations team for uplink to the spacecraft. 

- Generate orbit and acquisition data products. 

• On-Orbit Mission Operations Process (8) 

Perform routine on-orbit operations, special requests, and analysis as needed. Produce status reports as 
required. 

5.0 Building Proactive QA Into the Flight Dynamics Mission Support Process 

This section discusses FDCTs approach, the in-progress process reviews, and the payoff involved in building a 
proactive quality assurance process for GSFC Flight Dynamics mission support. 

5.1 Proactive QA Approach in the Form of Process Assurance 

The emphasis of the FDO’s QA approach is on proactive prevention of problems by the following activities: 

• Build quality checkpoints into FDO project plans, work package plans, mission support phase activity plans, 
and product QA plans. These checkpoints with appropriate quality criteria ensure that all pertinent 
activities are quality assured during the process before moving into the next phase of work. 

• Provide and/or assist in PPRs to project leader and POMs with just-in-time training to the technical staff 
before the start of a project or mission support phase and ensure awareness of quality checkpoints by 
technical staff. 

• Monitor the work and/or mission support phase processes and detect any deviations from the established 
processes and quality standards so that corrective action can be taken as early as possible in a mission or 
project life cycle. Regular internal mission reviews, IPPRs, preliminary reviews, dry runs before formal 
reviews, walkthroughs, and peer reviews are in this category. 

• Ensure compliance with all policies, standards, and procedures for all processes, products, and services. 

• Collect quality-related data, analyze, and feedback for process improvement. Coordinate with the FDO's 
Defect Causal Analysis (DCA) and Lessons Learned programs. 

A comprehensive QA approach is implemented at the FDO mission support project and task levels. An FDO 
mission support project is a task or a set of tasks involved in a given Flight Dynamics mission. An FDO mission 
support plan, prepared by a mission manager or project leader, defines all the major Flight Dynamics activities and 
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events that occur during the development and initial deployment of a mission. One example of this proactive 
approach is the conducting of IPPRs as described in Section 5.2. 

5.2 IPPRs for the FDO Mission Support Life Cycle 

The IPPRs are performed for all six phases of the FDO mission support life cycle. The IPPR assesses project team 
members’ understanding of and conformance to the S&Ps and provides an LCPA checklist for the mission-or 
project life cycle phase. The review occurs approximately 15 to 25 percent into the calendar schedule of the 
mission support life cycle phase. The PAO uses the mission phase-specific checklist to evaluate the phase activity 
progress. An assessment guideline for each phase has been set to an 80 percent or higher satisfaction level to 
determine whether the phase activities are on track. This guideline has proven to be achievable. 

Quality/process checklists were developed for the IPPR using the FDO mission support JIT training materials. The 
FDO Mission Support process coordinator and associates offer this JIT training to the mission teams before the 
beginning of each phase of the FDO mission support life cycle. PAO participates in training coordination. The 
training normally occurs 2 to 3 weeks before the start of each phase. These mission phase IPPR checklists are 
updated and enhanced periodically using the feedback comments from the project leaders, mission managers, team 
members, and lessons learned. Two sample IPPR checklists for the operations planning and training, and launch 
support phases are as shown in Tables 1 and 2, respectively. The questions labeled as PI, P2, and P3 were added to 
collect additional comments or suggestions for the process and overall assessment. 

5.3 Proactive QA and Payoff 

The quality approaches too often have been reactive rather than proactive in industry as a whole. Therefore, the 
cost has been high and the reactive QA also caused delays in the delivery schedule. The proactive quality approach, 
on the other hand, builds quality assurance into the work processes as the mission support life cycle progresses. 
While the reactive approach deals with the end products, the proactive approach deals with the processes, which 
prevents errors or rework. In this way, errors and inefficiencies can be detected earlier and corrected quickly, 
preventing excessive rework after a product is produced. Results of high-quality work from one phase will provide 
a sound foundation for the subsequent phase. The proactive quality assurance costs are small and repay well in 
quality products and services and timely delivery. Currently a little more than one PAO staff member has been 
supporting 150 FDO staff members effectively. The impact of process assurance activities on the quality of the final 
products and services could be considered minor; while the prime responsibilities are performed by the mission 
team. However, the cost savings via the process assurance could be significant to the multimillion-dollar space 
missions. 

6.0 Actual Implementation and Findings 

Over the past few years, CSC’s FDO has implemented a comprehensive proactive QA process applied to each of 
six phases of the Flight Dynamics mission support life cycle and the documents produced during each phase. As 
advocated, the quality approach has been proactive rather than reactive. In this way, errors, inefficiencies, and, 
therefore, rework have been prevented, and quality products and services were delivered on time. This subsection 
presents the status of how often and to what missions those proactive QA processes were applied (i.e., PPRs, POM, 
coordination of JIT training, internal mission reviews, dry runs or preliminary reviews before formal reviews, 
walkthroughs, audit coaching, IPPRs, and LCPA). It is difficult, if not impossible, to measure numerically the 
results or effectiveness of the processes. However, the proactive QA process has been well received by the line 
management and technical staff. It has served as a useful reminder on aspects of the process that can be forgotten 
in day-to-day work. The impression of the perceived improvement has been good as expressed by the mission 
managers. The FDO missions supported recently by this process were SAMPEX, Deep Space Program Science 
Experiment (DSPSE), Geostationary Operational Environmental Satellite (GOES) -I and -J, WIND, POLAR, 
X-ray Timing Explorer (XTE), and Solar and Heliospheric Observatory (SOHO) that were launched successfully, 
and Submillimeter Wave Astronomy Satellite (SWAS), Fast Auroral Snapshot Telescope (FAST), Total Ozone 
Mapping Spectrometer (TOMS), Tropical Rainfall Measurement Mission (TRMM), Advanced Composition 
Explorer (ACE), Wide-Field Infrared Explorer (WIRE), and Earth Observing System (EOS) that are to be 
launched in the near future. 


277 



Table 1. IPPR Checklist for the FDO Operations Planning and Training Phase 


Question 

Number 

Question 

1 

Have you established a training coordinator, a training plan, and discussed the training plan with 
your ATR(s)? 

2 

Has the team begun drafting procedures and is there a central repository for these procedures? 

3 

Has a mechanism been established to track action items? 

4 

Has a readiness matrix been prepared? (Coordinate with the Flight Dynamics Institution of 
Simulation and Training (FIST) group.) 

5 

Have you contacted the mission coordinator concerning the development of the Mission Support 
Plan (MSP)? 

6 

Were any lessons learned recorded for this phase? 

7 

Have any software problems that were detected (during use for training and simulations) been 
documented and provided to the development/maintenance personnel? What is the resolution 
status? 

8 

What will be the products produced and Standards & Procedures (S&Ps) applied? 

PI 

Was the JIT training adequate to start the phase? 

P2 

Any other problems encountered in this phase or suggestions? 

P3 

Overall Assessment: 

P. 1 Is the phase support on track? 
P.2 If not, what is the resolution? 


Table 2. IPPR Checklist for the Launch Support Phase 


Question 

Number 

Question 

1 

Who is the central contact point during the launch support? 

2 

What is the chain of reporting in the Flight Dynamics Facility (FDF) for emergency (within CSC and 
with the GSFC chain)? 

3 

What are the launch support components, group leaders and members, and communications 
required between the component groups? 

4 

What shift groups are there, who are the members and leaders, and are phone numbers available? 

5 

What are the unique nature and requirements of this launch support? 

6 

What are recent procedure changes and terms are not familiar to everybody on the team? 

7 

What are the interfaces external to FDF, communications/data transfers required, and different terms 
used? 

8 

What are the potential problems or mishaps and resolution (or mitigation) strategy? Do all the team 
members know how to respond to those emergencies or contingencies? 

9 

Have you obtained any lessons learned from the previous missions and team members for this 
mission support? 

10 

Are there any problems or pending items that make the team uneasy at this point? 

PI 

Was the JIT training adequate to start the phase? 

P2 

Any other problems encountered in this phase or suggestions? 

P3 

Overall Assessment: 

P.1 Is the phase support on track? 
P.2 If not, what is the resolution? 
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The examples in Table 3 show various Table 3. FDO In-Progress Reviews for the XTE Missions 

types of in-progress reviews conducted at 
different stages of the life cycle phases for 
the XTE mission. An IPPR was not con- 
ducted for every phase of the entire 
mission life cycle. The reviews supported 
were performed only when PAO had 
available time; therefore, the lists are not 
as complete as they could be. 

In addition to the reviews listed in 
Table 3, FDO internal mission reviews are 
performed regularly (for example, once or 
twice a month with the FDO management 
until launches). The PAO attends these 
review meetings. After IPPRs, DORRs, 

LCPAs, and other reviews, PAO writes 
review reports and distributes copies to the project or mission team leader. Lessons learned data are used for 
process improvement and are disseminated to the next mission teams. The project or mission managers are 
interviewed before and after the launch to evaluate the impacts of PAO’s reviews. They have made the following 
observations: 

• The FDO PAC procedure provides a comprehensive reminder list for the phase activities while the team 
members are otherwise occupied. 

• The PAO review motivates FDO mission manager and team members to move on to the next phase. 
Occasionally, the team experience some difficulty starting up a new activity following the completion of a 
phase. 

• The review educates new mission managers who have not managed the entire mission support life cycle. 

• JIT training before the phase was useful to mission managers and team members. 

• Coordination or providing lessons learned for the mission manager and team was helpful. 

• Knowing ahead of time about the lessons learned from the previous missions was helpful for the mission 
manager and team when planning future resolutions. 

• The prelaunch readiness review meeting improved the morale of the team and helped team building. 

The following were lessons learned: 

• JIT training needs to be done at the right time (but it is difficult to define that time), and JIT needs to be 
repeated or spread throughout the phase with different emphasis; overview, details, and refresher. 

• Deployment of the process was suggested to all members of the organization rather than limited to mission 
managers or project leaders. 

The survey continues and feedback comments are collected in FDO for both the process itself and associated 
checklists. The same in-progress process assurance principle as used in the mission support phase IPPR has been 
applied to FDO document quality via the Product QA Plan. In the Product QA Plan, quality checkpoints are 
identified and appropriate quality criteria are provided ahead of time for QA reviewers. 

7.0 Document Reviews and FDO Product QA Plan 

Another aspect of FDO process assurance is to ensure that documents produced are quality assured and to provide 
the QA reviewers some guidance on how to perform the QA. One of the primary approaches to achieving technical 
quality in products is the FDO Product QA plan. Products include documents, memoranda, technical papers, and 
presentation materials. The Product QA Plan form for each FDO deliverable product includes key steps that verify 


Review 

Date 

Requirements and Specifications Phase IPPR 

04/08/93 

Software Development Support Phase IPPR 

09/01/93 

Project IPPR 

04/21/95 

DORR (L-60) 

06/29/95 

DORR (L-30) 

07/27/95 

Launch Phase PPR 

07/28/95 

Launch Phase POM 

07/31/95, 

12/05/95 

Operations Planning and Training Phase IPPR 

11/21/95 

XTE Launch 

12/30/95 

Lessons-Learned Workshop 

03/29/96 
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whether an appropriate QA reviewer has been selected, S&Ps have been identified, QA checkpoints have been 
identified with appropriate QA checklists (both generic and document-specific) attached, coordination with FDD 
Configuration Management Office (CMO) and Technical Publications has been scheduled, and QA data collection 
has been planned for analysis. 

The QA reviewer may be anyone in the FDO who is technically qualified to perform the review and who does not 
report administratively to the line manager responsible for the product. The intent of this review is to ensure the 
technical quality of products, but their appearance and adherence to schedule may also be reviewed. 

In the FDO Product QA Plans, key quality review steps or checkpoints are identified and the corresponding quality 
criteria are attached for document reviews such as for system and operations concepts, requirements specifications, 
functional specifications, mathematical specifications, presentation slides for SOCR, SRR, and DORR, mission 
support plan, analysis reports, professional paper, etc. The QA reviewers then use the quality criteria to review the 
documents. The Product QA Plan form and associated QA checklists have been updated and enhanced periodically 
using the feedback comments from the QA reviewers and authors, and lessons learned. Historically, lists of 
document quality criteria were developed in FDO earlier than the IPPR checklists for the Flight Dynamics mission 
support life cycle phases. Some of the document quality criteria were used to generate phase IPPR checklists as 
mentioned earlier in the paper. 

An example of specific quality criteria for a requirements specifications document (used with an FDO Product QA 
Plan) is shown in Table 4. In the document review process, general QA categories are also incorporated such as for 
technical accuracy, internal consistency, conciseness, freedom from redundancy, clarity, and adherence to the 
established standards and format. 

Table 4. Quality Criteria for the Requirements Specifications Document 


Criteria 

Number 

Criteria 

1 

Are the DMR and related studies referenced when necessary? 

2 

Are all requirements defined for all mission phases? 

3 

Are the assumptions reasonable and precisely stated? 

4 

Does the requirements summary correctly trace requirements to a high level source document? 

5 

Does the requirements summary include all requirements critical to the definition of the system 
and operations concepts? 

6 

Are items such as major external interfaces summarized? 

7 

Is there consistency between the high-level and low-level requirements? 

8 

Does the document provide complete information to facilitate the following: 

• The analysts in preparing functional specifications? 

• The analytical and acceptance test teams in preparing test plans? 

9 

Other (Specify) 


8.0 Process improvement Through Lessons Learned 

Each mission team within the FDO is responsible for developing and documenting improvements and continuing 
good practices based on the lessons learned during day-to-day activities or during the generation of products and 
services. The vehicles used to deploy lessons learned are the project lessons learned database, the FDO lessons 
learned database, reports, electronic files or bulletin boards, training, and DCA reports. These lessons learned are 
fed back into the process for continuous improvement. The PAO identifies and evaluates S&Ps that are appropriate 
for FDO products and services. It also establishes and/or assists in developing S&Ps for FDO products and services 
when appropriate S&Ps do not already exist. The PAO promotes communication among FDO projects and/or 
missions on tools and methodologies that may contribute to improved quality and productivity. A continual effort is 
made to improve process assurance and quality assurance methodologies for the FDO-unique mission analysis and 
support activities. Process and QA methodology changes include those due to changes in mission requirements, 
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institutional changes, and lessons learned. Methodology effectiveness is closely monitored and evaluated for 
improvement. 

A few examples of the lessons learned that contributed to the FDO process improvement are as follows. The IPPR 
checklist for the launch support phase (Table 2) was improved after it was learned that other NASA centers, 
ground stations, and the European Space Agency (ESA) often use different terms from those used at GSFC. A 
prelaunch readiness review or phase orientation meeting with the PAO-generated review item check list was very 
positive in team building and morale boosting for the DSPSE mission. Since then, the GOES-I, GOES-J, WIND, 
SOHO, XTE, and POLAR mission teams used this process Concerning FDO document reviews, it was observed 
(in general) that the review results vary depending on the QA reviewers even after a review checklist was given 
ahead of time. Document review skills training is being considered in FDO. 

9.0 Summary and Recommendations 

The FDO PAC procedure incorporates process/quality checklists for each of the six FDO mission support life cycle 
phases. The findings from the actual implementation of the phase IPPRs showed that perceived quality 
improvement and feedback comments have been very positive. The intermediate results have been good. For 
further improvement to the FDO process assurance, it is recommended to (1) implement all in-progress process 
reviews over the entire life cycle from concepts to on-orbit mission operation phase, (2) devise a scheme to measure 
the cost of this process compared with the total mission cost, and (3) adapt the FDO PAC model to the needs of 
evolving technology — commercial off-the-shelf software, personal computers and/or workstations, and joint 
application development (JAD) or rapid application development (RAD) approaches. In the areas of FDO product 
QA, more training must be provided for authors as well as QA reviewers concerning the QA checkpoints, quality 
criteria and review skills. The proven processes and lessons learned need to be promulgated throughout FDO. 
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